Skip to content

Keep "the last day" as the last 24 hours when a Bot hands work to a task - #1730

Merged
davidmckayv merged 1 commit into
mainfrom
fix/last-day-window
Nov 16, 2026
Merged

davidmckayv merged 1 commit into
mainfrom
fix/last-day-window

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

The bug

Asked "go through my inbox for emails from customers in the last day", a Bot on the Claude Agent SDK handed the work to a task with a brief that often said "since 2026-11-15" rather than "the last 24 hours". The task did what the brief said: it searched Gmail with after:2026/11/15, which takes whole calendar days, got back an email 30 hours old, and put it in the sheet with the day's three customers. This happened in about 1 run in 4. Bots on the built-in harness (Claude and GPT) and on LangGraph wrote "the last 24 hours" and never did it.

The Gmail search guide already says to use newer_than:1d and to check each message's time, and a result searched by date already says it covers whole days. Neither helped, because the window the task was given was the calendar date, and the 30-hour-old email really was "since 2026-11-15".

The fix

One sentence in the delegating guidance, beside "every figure, name, link and constraint ... the work needs":

Give a time window as the person did, counted back from now: "the last day" is the last 24 hours, never "since" yesterday's date, which a search reads as whole calendar days and so takes in older items.

Test

shared/tests/orchestration.test.ts, "a brief keeps a time window counted back from now, never a calendar date". Red on main (1 fail), green with the fix (361 pass, 0 fail across shared).

Measured offline (captured runs replayed on the real model, 6 times each)

Step replayed Main With this change
Claude SDK, writing the brief (4 captured runs, 24 replays): briefs that anchor the window on a date 12 of 24 0 of 24
Claude SDK task, writing the rows after a search returned the 30-hour-old email, brief as main wrote it vs. as this change writes it (3 captures) 18 of 18 held it 0 of 17 held it (one replay wrote no rows yet)
Built-in Claude, built-in GPT, LangGraph, writing the brief (18 replays) 18 of 18 rolling 24 hours 18 of 18 rolling 24 hours

Live results follow in a comment.

@davidmckayv

Copy link
Copy Markdown
Contributor Author

Live

The multi-step task ("emails from customers in the last day" into a sheet, then a post in #sales), real models, on two AWS boxes each with its own local Intelligence (0.16.0-rc.13): one on the code before this change, one with it. Both judged by a check that fails a sheet holding mail older than the last 24 hours.

Setup Runs Mail older than the last day in the sheet, before After Briefs anchored on a date, before After
Claude Agent SDK 6 2 of 6 0 of 6 3 of 6 0 of 6
Built-in, Claude 3 0 of 3 0 of 3 0 0
Built-in, GPT 3 0 of 3 0 of 3 0 0
LangGraph 3 0 of 3 0 of 3 0 0

Nothing else moved with the change. Claude's final answers ran over the 120-word limit before and after (that is the separate answer-length work), and the odd model-graded miss appears on both sides. Three runs met a Slack action the test's fake apps don't play, so they are invalid, not failed.

Asked for customer mail "in the last day", a Bot on the Claude Agent SDK wrote the task's brief as
"since 2026-11-15" in half of its replays. The task then searched by that date, which takes whole
calendar days, and filed a 30-hour-old email as one of the last day's. Bots on the other harnesses
already wrote "the last 24 hours".

The delegating guidance now says to give a time window as the person did, counted back from now,
never as "since" yesterday's date.
@davidmckayv
davidmckayv merged commit 01e5460 into main Nov 16, 2026
16 checks passed
@davidmckayv
davidmckayv deleted the fix/last-day-window branch November 16, 2026 20:50
davidmckayv added a commit that referenced this pull request Nov 16, 2026
…, no list, one for what to check (#1726)

Claude Sonnet 5.5 reported a three-app job in 166 to 252 words, on OpenBot's own runner and on the Claude Agent SDK alike, where GPT-5.6 Terra took 45 to 84: the sheet's rows again, "I read the sheet back", what it left out and why, and each inference as a bullet. It follows the task's brief, which comes last and often asks for "anything you could not confirm", over a rule before it.

- After the brief, a task is told how a report of work done is shaped: one paragraph of at most four sentences, no list, what was done with its link, then at most one sentence on what to check, with a worked example. A question's answer is shaped as before. readBrief takes the text off again, and a thread opened without it still reads back.
- The reply rules say a finished job the same way; the claims rule adds that an inference, however likely, is still unconfirmed.
- A delegating Bot's brief is not told to stop short: a line that did so made Claude drop the rolling 24 hours from its briefs, and a test keeps it out.
- The Claude Agent SDK fixture of what the server sends is rewritten, and the harness test checks Claude Code starts with the new rules.

Live, multi-step case, Claude Agent SDK, 6 runs each on main with #1730 and on this change: did the task 6 and 6, claims 5 and 5, words 177-270 and 78-140.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant